Derivation and application of color accents for images in a video delivery system interface

ABSTRACT

In one embodiment, a method receives an image of video content for a video offered by a video delivery service. The method analyzes pixels in the image to determine color attributes of the pixels, the color attributes including a lightness attribute and a hue attribute. Lightness counts for the image based on the lightness attribute and hue group counts based on the hue attribute are generated for the pixels. The method selects a lightness classification from lightness classifications based on the lightness counts and selects a hue group from hue groups based on the hue group counts. The lightness classification and the hue group classification are sent to a client device to allow the client device to generate a color using the lightness classification, the hue group classification, and a saturation value, the color being applied to an interface on the client device that is displaying the image.

BACKGROUND

An application may display different images in a user interface. Toprovide a pleasing experience to the users, the application may usecolor accents to bridge the gap between the images and portions of theuser interface. For example, depending on the image displayed, certainaspects of the user interface may be adjusted. In one example, for eachimage a client receives, the client can analyze the image and selectwhich color accents to use for the user interface. This may beconvenient because different clients may have different specificationsfor generating color. However, allowing the clients to perform theanalysis when an image is received may provide different experiences forusers of different devices. Further, having the client analyze the imagewhen the image is received may be time-consuming and delay the displayof the user interface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a simplified system for displaying images with coloraccents in a user interface 106 according to one embodiment.

FIG. 2 depicts a simplified flowchart of a method for generating theimage metadata according to one embodiment.

FIG. 3 depicts a simplified flowchart of a method for determining colorsto apply to a user interface in conjunction with displaying an imageaccording to one embodiment.

FIG. 4 depicts an example of a user interface according to oneembodiment.

FIG. 5 depicts a video streaming system in communication with multipleclient devices via one or more communication networks according to oneembodiment.

FIG. 6 depicts a diagrammatic view of an apparatus for viewing videocontent and advertisements.

DETAILED DESCRIPTION

Described herein are techniques for an image analysis system. In thefollowing description, for purposes of explanation, numerous examplesand specific details are set forth in order to provide a thoroughunderstanding of particular embodiments. Particular embodiments asdefined by the claims may include some or all of the features in theseexamples alone or in combination with other features described below,and may further include modifications and equivalents of the featuresand concepts described herein.

Particular embodiments generate image metadata for images that are goingto be displayed on a user interface. The image is analyzed to generate alightness group classification and a hue group classification based oncolor attributes of pixels in the image. The lightness group and huegroup classifications are then stored for the image as image metadata.When a client device receives a selection from a user in the userinterface that causes display of the image, the application can use theimage metadata to generate one or more colors (e.g., color accents) thatare applied to the user interface. For example, a color is generatedbased on lightness value and saturation value that are determined basedon the lightness group classification and a hue value that is determinedbased on the hue group classification. If multiple colors are desired,different lightness values and saturation values may be determined basedon the lightness group classification and used with the same hue valueto generate multiple colors. The generated colors can then be used toaccent portions of the user interface when the image is displayed.

Overview

FIG. 1 depicts a simplified system 100 for displaying images with coloraccents in a user interface 106 according to one embodiment. System 100includes a video delivery system 102 and client devices 104. Videodelivery system 102 may deliver videos to client devices 104 via a videodelivery service. Client devices 104 may display user interface 106 foran application that allows a user to browse and select videos to play.Because user interface 106 is for a video delivery service, theapplication may display images from content of the videos on userinterface 106. For example, when a user selects a video title, userinterface 106 displays an image or a short video clip from the video(e.g., an image from a movie or a scene from the movie). As differentimages are displayed, the look and feel of user interface 106 may changebased on the content of the different images being displayed. Particularembodiments change a color or more than one color that is applied inuser interface 106 based on the image currently being displayed.

As a user browses, a display process 108 in client device 104 generatesuser interface 106. User interface 106 may display different content,such as menus, images, and accent areas. At different times, displayprocess 108 may display different images, such as different images fromdifferent videos. An image from the video may be a single image or aseries of images (e.g., an animation of multiple images of five seconds,or a short period of video). The different images from the videos may bedifferent in that they may have different color attributes, such aslightness, hue, or saturation. To provide a unified experience to users,display process 108 may generate colors that are used to accent userinterface 106. For example, portions of user interface 106 may begenerated using colors that are based on the content of the image. Theseportions may include a selection indicator, a border area, a backgroundarea, or other portions of user interface 106. The colors may be appliedto user interface 106 in different ways, such as displayed as is on userinterface 106 or may be used to generate different colors on userinterface 106. Accordingly, as different images are displayed, displayprocess 108 may change the color that is used for portions of userinterface 106.

Before client device 104 can display the colors in user interface 106,video delivery system 102 processes the images that will be displayed inuser interface 106 to generate image metadata. Then, display process 108uses that image metadata to generate the colors for the portions of userinterface 106 when an image is displayed.

To generate the image metadata, an image processor 110 receives imagesthat could be displayed in user interface 106. For example, the imagesmay be received from videos that are offered in the video deliveryservice by video delivery system 102. Image processor 110 then analyzesthe image to generate color attributes and generates the image metadatafrom the color attributes. In one embodiment, the image metadata mayinclude a hue classification and a lightness classification for theimage. A hue may be a degree representing the color hue to use whengenerating colors using a HSL (hue, saturation, and lightness orluminosity) representation or HSV (hue, saturation, and value)representation in a RGB (red, green, blue) color model. In oneembodiment, a cylinder may represent the HSL or HSV representation. Ineach cylinder, the angle around a central vertical axis represents thehue, the distance from the axis corresponds to saturation, and thedistance along the axis corresponds to lightness or value. Fordiscussion purposes, particular embodiments will use lightness, butlightness could also be derived from the value in HSV. When lightness isused, a person of skill in the art will recognize, the attributeequivalently exists in HSL and HSV representations. In one embodiment,image processor 110 analyzes the pixels to generate the hueclassification and the lightness classification, but not the saturation.In one embodiment, the saturation is defined with a lightness value fora lightness classification. The hue classification informs the purecolor base that forms the actual visual color. The lightnessclassification provides an indicator to ensure that colors contrastproperly with the images by informing the final saturation/lightness orsaturation/value pairs that are used to generate actual colors. Imageprocessor 110 then stores the image metadata in storage 112 for lateruse when a client device 104 generates user interface 106.

After generating the image metadata, an interface generator 114 receivesrequests from client device 104 for generating user interface 106. Inresponse, interface generator 114 may send an image to client device104. Also, interface generator 114 sends the image metadata for theimage. As discussed above, the image metadata may include a hueclassification and a lightness classification. In one embodiment, onlythe hue classification and the lightness classification are sent asimage metadata and not a saturation value. Further, the red, green,blue, and/or alpha components that make up an RGBA color are not sent toclient device 104 in the image metadata. Although RGBA color isdescribed, the alpha component may not be used, and only an RGB color isused. It is noted that the image and image metadata may be sent atdifferent times. For example, the image and image metadata may be sentin real-time in response to a request. In another embodiment, the imagemetadata may be sent before the request. In other embodiments, theimages and image metadata may be downloaded to client device 104 andstored at client device 104.

Display process 108 receives the image and the image metadata and canthen generate user interface 106. In generating user interface 106,display process 108 uses the hue classification and lightnessclassification to generate a color, such as an accent color, forportions of user interface 106, a more detailed process of which will bedescribed below. Display process 108 could generate a single color ormultiple colors from the hue classification and lightnessclassification. Then, display process 108 generates user interface 106,which includes the image and the portions accented with the color. Inthis case, display process 108 did not analyze the image to generate thered, green, blue, and alpha components of the RGBA color. Rather,display process 108 performed HSL and/or HSV color space transformationsusing the lightness classification and hue classification to generatethe RGBA color to use for the portions.

Image Metadata Generation

FIG. 2 depicts a simplified flowchart 200 of a method for generating theimage metadata according to one embodiment. The process described inFIG. 2 may be performed for every image that may be displayed by userinterface 106. At 202, image processor 110 receives an image. Forexample, the image may be received when a new video is received by videodelivery system 102 and video delivery system 102 selects an image fromthe video.

At 204, image processor 110 analyzes a pixel of the image to determinecolor attributes for the pixel. Although a pixel is described, a groupof pixels may be analyzed. Also, in some cases, every pixel of the imagemay be inspected or just a portion of the pixels. Optimizations of theprocess will be described in more detail below.

For the pixel, at 206, image processor 110 updates a lightness groupcount based on the color attributes of the pixel. For example, there maybe multiple lightness groups in which the pixel could be classified,such as a dark classification, a medium classification, and a lightclassification. For the hue group, degree slices of a certain size ofhue definitions in the HSL and/or HSV color spaces may be defined, suchas a slice size of five degrees. Using a slice size of five degreesmeans a hue group includes five degrees, such as hue group #10 containsdegrees 10 through 14 inclusive and hue group #15 may include degrees15-19 inclusive. Inclusive means hue group #10 includes degrees 10, 11,12, 13, and 14. It will be understood that other ranges may be used. Inone embodiment, the hue groups may be from 0-360 degrees.

In one embodiment, the process to generate the lightness group and huegroup counts may be as follows. First, image processor 110 determinesthe color attributes as the red, green, blue, and/or alpha components ofthe pixel that make up an RGBA color. Not all of the RGBA attributes maybe determines or other color attributes may also be used. From the RGBAcolor, image processor 110 derives the hue, chroma, saturation, and/orlightness attributes. In one embodiment, the hue, chroma, saturation,and lightness attributes are as defined by HSL and/or HSV color spacetransformations. The hue attribute is an attribute of visual sensationwhere an area appears to be similar to one of red, yellow, green, andblue or to a combination of two of them. The chroma is a colorfulnessrelative to the brightness of a similarly-illuminated white color. Asaturation is the colorfulness of a stimulus relative to its ownbrightness. The lightness is the brightness relative to the brightnessof a similarly-illuminated light.

Image processor 110 may then define light and dark thresholds. Forexample, the light threshold and the dark threshold may be lightnessvalues. Image processor 110 compares the lightness value to the darkthreshold and the light threshold to determine the lightness groupclassification. For example, if the lightness value is less than orequal to the dark threshold, image processor 110 increments the darkclassification for the lightness group. If the lightness value isgreater than the dark threshold and less than the light threshold, thenimage processor 110 increments the medium classification. If thelightness value is greater than or equal to the light threshold, thenimage processor 110 increments a lightness classification. Although thisprocess is described, other processes may be used to determine thelightness classification.

At 208, image processor 110 then determines if the color should beconsidered in the hue classification. For example, in some cases, if allpixels were used in the hue classification, then the hue that isdetermined may not generate a desired color for the portions of userinterface 106. Image processor 110 removes pixels with colors that arenot considered desirable to use to classify the hue. In one embodiment,image processor 110 determines whether the color should be used usingconfigured color attribute thresholds. In one example, image processor110 ensures that the chroma falls inclusively within a configured chromarange, ensures saturation falls inclusively within a configuredsaturation range, ensures lightness falls inclusively within aconfigured lightness range, and ensures the maximum distance between theR, G, and B components of the color exceeds a configured minimum neutraldistance. The neutral distance is the distance a color is from a puregray. This ensures the color is far enough from gray to not be perceivedas one. If all of the thresholds are met for the color (or a portion ofthem), at 210, then image processor 110 determines that the color shouldbe considered for hue classification and increments the count of the huegroup whose slice contains the color's hue value. For example, if thehue is 11°, then the count for hue group #10 is incremented. If thecolor should not be considered for the hue classification, then imageprocessor 110 does not increment any hue group counts.

Once the lightness group and hue group counts are updated, at 212, imageprocessor 110 determines if another pixel should be analyzed for theimage. If so, the process reiterates to 202 where another pixel isinspected.

After all the pixels within the image are inspected, at 214, imageprocessor 110 analyzes the lightness group count and the hue group countto select the final classifications for the image. For example, thelightness count that is the largest is selected as the lightnessclassification for the image. In this case, the lightness classificationfor the image could be either light, medium, or dark. The hueclassification of the image is the hue of the hue group which count isthe largest. In one embodiment, image processor 110 may apply a biasingfunction to the hue group counts. The biasing function operates on thedomain of all possible hue values and returns a multiplier that shouldbe used to manipulate the hue group counts. This function allows forcertain parts of the color spectrum to be favored over others to achievedesired results, such as design cohesion, and avoid jarring/displeasingcolors. The biasing function may be determined based on the videodelivery system's desires in providing the user interface.

At 214, image processor 110 stores the selected lightness classificationand hue classification for the image. The image metadata may be storedalong with other metadata for later transmission to client devices 104.

In one embodiment, swatch configurations may be used that defineswatches. Each swatch may be a color that could be applied to a portionof user interface 106. In one embodiment, each swatch configuration maydefine three swatches that produce three different colors for eachlightness classification. Also, the hue value used may change for eachdifferent color. For example, for the light, medium, and dark lightnessclassifications, image processor 110 defines hue (H), lightness (L), andsaturation (S) values. In one example, the values may be:

-   Light lightness classification: H=X, S=30, L=20;-   Medium lightness classification: H=X, S=50, L=40; and-   Dark lightness classification: H=X, S=70, L=60.

If multiple colors are generated for one image, then at least one of thehue, saturation, and/or lightness may be varied within a classificationto create another color. For example, the medium lightnessclassification may have a second color of H=X+20, S=50, L=40 or H=X,S=60, L=50, and a third color of H=X+40, S=50, L=40. In one example,when an image is classified with a medium lightness and hue group #10,then three colors are generated as [H=10, S=50, L=40], [H=30, S=50,L=40], and [H=40, S=50, L=40].

In one embodiment, image processor 110 may generate the different hue,saturation, and lightness values for a lightness classification and huegroup classification. These values may be generated based on a context,such as characteristics of client devices, or the values may bepre-defined. Video delivery system 102 then sends these values to clientdevices 104. In another embodiment, client devices 104 may use thelightness classification and hue group classification to generate thedifferent hue, saturation, and lightness values.

Image processor 110 may also use optimizations when analyzing an image.The optimizations may leverage the principle that the exact color of thepixel does not matter; only the hue group it belongs to matters. Incontent images, such as images from a video, pixels directly adjacent toeach other are either the same or slight variations in the majority ofcases. Hue group slices are generally large enough to encompass slightvariations of adjacent pixels. Pixels adjacent to each other areunlikely to change enough and could be excluded if the previous pixelwas excluded. Sudden changes in pixels caused by hard lines in imageswill contain new segments of similar adjacent pixels. Images composedsolely of extreme pixel changes are very unlikely to be chosen as animage by the video delivery system for the application being displayedin user interface 106. Using these observations, image processor 110 canincrease performance by only analyzing a subset of pixels instead ofevery pixel in the image. Image processor 110 may determine the numberof pixels that can be skipped based on a function of the size of the hueslice size used to define the hue groups. In one embodiment, everyincrease of five degrees in the hue groups allows another 100 pixels tobe skipped per pixel analyzed. The performance increase from theoptimization may produce between five and ten times the performancegain.

Client Processing of Image Metadata

The image metadata may be provided to client devices 104 in differentways. For example, image metadata may be sent to a client device whenthe client device logs on to the video delivery system. Also, the imagemetadata may be sent along with the image when requested by clientdevice 104. In one embodiment, the image metadata is sent via a channelfrom video delivery system 102 to client device 104 that is either usedto send the image or is a separate channel.

FIG. 3 depicts a simplified flowchart 300 of a method for determiningcolors to apply to user interface 106 in conjunction with displaying animage according to one embodiment. At 302, display process 108 at aclient device 104 receives a user input on user interface 106 thatcauses display of a new image. For example, a user navigation may selectvarious items on user interface 106. In one example, the items may bedifferent video titles. When a video title is selected, display process108 will display an image for that video.

At 304, display process 108 sends a request for the image to videodelivery system 102. In another embodiment, display process 108 may havereceived the images already from video delivery system 102 and storedthe images in storage on client device 104. In this case, a request foreach image does not need to be sent to video delivery system 102.

At 306, display process 108 receives the image and image metadata fromvideo delivery system 102. The image metadata may be received inreal-time when the user input is received. Or, the image metadata may beretrieved from storage from client device 104. Also, the image metadatamay be automatically sent when an image is requested and client device104 does not need to request the image metadata separately.

At 308, display process 108 generates a color from the image metadata. Asingle color will be described, but it is recognized that multiplecolors may be generated from the same image metadata. One advantage toproviding lightness and hue classifications is that client device 104can apply different saturation and lightness values to generate multiplecolors. If an RGBA color was provided, client device 104 would not beable to directly use that color to generate other colors. Rather, clientdevice 104 would need to convert to the HSV or HSL color space. Theprocess to generate the color or colors will be described in more detailbelow.

At 310, display process 108 generates user interface 106 using the colorand the image. For example, the accent portions of user interface 106may be colored using the color and the image may be displayed in userinterface 106. For example, if a movie is selected, a content image fromthe movie is displayed in the background of user interface 106 and thecolor is applied to certain portions of user interface 106. In oneembodiment, a selector used to perform the navigation is accented usingthe color. Other portions of user interface 106, such as the border, mayalso be accented using the color. Further, the background portion ofuser interface 106 that is displaying the image may be blended using thecolor.

Different processes may be used to generate the color or colors. In oneembodiment, display process 108 receives swatch configurations. Theswatch configurations define different saturation values and lightnessvalues to use for each lightness classification. The swatchconfigurations could be the same for all clients 104, different fordifferent types of clients 104, or personalized per user. In oneembodiment, the swatch configurations may be received and stored atclient device 104 at different times. For example, before the requestfor the image is sent, client device 104 may have already received theswatch configurations via a configuration request or secondary serverrequest. For each swatch configuration, the saturation and lightnessvalues are defined for each lightness classification and are the samefor all hues.

The saturation values and lightness values can be combined with the hueclassifications for the image to generate one or more colors to beapplied to user interface 106. For example, the hue value used may bethe base hue classification, such as if the hue classification is huegroup #10 for degrees 10-14, then the hue value of 10 is used, but othervalues could be used within the range (e.g., 11, 12, 13, or 14).Similarly, the hue value of 15 is used when hue group #15 is usedbecause hue group #15 includes the degrees of 15-19 inclusive. Anotherhue value may also be used within the group if desired. When displayprocess 108 loads an image that has related image metadata for coloraccenting of user interface 106, display process 108 uses the hue andthe swatch configuration appropriate for the lightness classification tocreate the colors that are applied to user interface 106. In oneembodiment, different variations of colors are used to accent userinterface 106, but are derived from the same hue classification assignedto the image. In one example, if the hue classification is 170, threecolor swatches are generated by applying three different sets ofsaturation values and lightness values to the hue value of 170 togenerate three HSL colors. The different colors are then applied toportions of user interface 106. In one embodiment, a first color is usedon a first portion of the border of the page and a second color is usedon a second portion of the border of the page. A third color may beblended into a background and used to accent the image. The third colormay mirror the tone of the image that is used as a background gradient.

Display process 108 may operate in the HSL and/or the HSV color spacenatively. In this case, RGBA values are not provided from video deliverysystem 102 to client device 104 for each image. If RGBA values wereprovided, then multiple RGBA values would have to be provided for eachof the different colors used for user interface 106. This may use a lotof bandwidth. Or, a conversion from the RGBA value to the HSL and/or HSVcolor space would have to be performed, then modifications applied inthe HSL and/or HSV color space to create the different colors, and thena conversion back to RGBA would be needed. This conversion may beimprecise due to the nature of the conversions and could cause userinterface 106 to look different across client devices 106. Further,depending on how the client is displaying the image, the lightness ofthe image on the client may need to be analyzed, but clients may not beable to determine the lightness of the image. Accordingly, the onlysolution if the RGBA space is used might be to send all three RGBAvalues to client device 104. However, working in the HSL and/or HSVspace, only the hue classification and the lightness classification needto be sent using particular embodiments. The swatch configurationincludes the lightness values and the saturation values that are neededto generate the colors.

When display process 108 needs to compute the colors to display on userinterface 106, display process 108 retrieves the image metadata that wasgenerated by image processor 110 for the image. This includes thelightness classification and hue classification. Then, display process108 selects the swatch configuration values that are associated with theimage metadata. In one embodiment, client device 104 receives the swatchconfiguration from video delivery system 102. In other embodiments,client device 104 may generate the swatch configuration itself, such asby applying a display context (e.g., a theme) of client device 104 togenerate the swatch configuration. Display process 108 selects thesaturation value and the lightness value for the lightnessclassification from the swatch configuration. Then, for each swatch,display process 108 generates a color using the saturation and lightnessvalue and a hue value. In one embodiment, a hue adjustment may be usedin one or more of the swatches to adjust the hue value. The hueadjustment changes the hue value used so that different swatches mayhave different base colors. It is noted that some swatches or none mayuse the hue adjustment. For example, some swatches may just usedifferent lightness and saturation values with the same hue value. Inanother example, the same lightness and saturation values are used withdifferent hue values, or all three of the lightness, saturation, and huevalues may be different for the swatches. The hue adjustment is a valuethat may be added to the hue when constructing the HSL and/or HSV color.For example, if the hue adjustment is 80 and the hue value is 50, thenthe adjusted hue value is 130.

User Interface

FIG. 4 depicts an example of user interface 106 according to oneembodiment. On user interface 106, a user may use a selector 402 toselect different items displayed on user interface 106. For example,items may include categories, such as category #1 403-1 and category #2403-2, and also specific items within each category, such as entity #1402-1, entity #2 402-2, and entity #3 402-3. The categories maycategorize items and the entities may be videos that could be requestedby the user for playback on user interface 106. When selector 402 isnavigated to different items, such as categories and/or videos, an image404 is displayed on user interface 106, which may be displayed in thebackground of user interface 106. For example, selector 402 has beenused to select entity #1 and an image from the video associated withentity #1 is displayed as the image.

In one embodiment, different portions of user interface 106 are accentedwith different colors that are generated as described above. Asdiscussed above, a single hue classification is used to generate threedifferent colors. In one example, a first portion 406-1 of userinterface 106, such as top left and bottom right border areas, may beaccented with a first color #1. A second portion 406-2, such as topright and bottom left border areas, may be accented using a second color#2. Further, a third portion 406-3 may be accented using a color #3.Color #3 may be used to blend the background of the image. Also,selector 402 may be accented, such as color #1 and color #2 may becombined in selector 402. It may be important to accent selector 402because this is the selector that is used by the user to selectdifferent items in user interface 106. Because a different image isbeing displayed in user interface 106 for different selections of theitems, the change in accent color for selector 402 is important toprovide a clear indication to the user which item has been selected.Although these portions are described as being accented with thesecolors, it will be understood that other portions may be highlighted.Also, different combinations of colors may be used in differentportions.

Accordingly, particular embodiments provide a process that can selectimage metadata that can be used to accent user interface 106 that isdisplaying the image. The process for generating the image metadata maygenerate a hue classification and light classification that can be sentto client devices 104. This allows client device 104 to not have toanalyze the image. Further, the single hue classification and lightnessclassification for an image can be used by each client device 104 togenerate the accent colors to be used in user interface 106. Instead ofhaving to send individual RGBA colors to client device 104, clientdevices 104 can generate these colors themselves. This may make thedisplay of page 400 to be uniform on multiple client devices 104.

System

Features and aspects as disclosed herein may be implemented inconjunction with a video streaming system 500 in communication withmultiple client devices via one or more communication networks as shownin FIG. 5. Aspects of the video streaming system 500 are describedmerely to provide an example of an application for enabling distributionand delivery of content prepared according to the present disclosure. Itshould be appreciated that the present technology is not limited tostreaming video applications, and may be adapted for other applicationsand delivery mechanisms.

In one embodiment, a media program provider may include a library ofmedia programs. For example, the media programs may be aggregated andprovided through a site (e.g., Website), application, or browser. A usercan access the media program provider's site or application and requestmedia programs. The user may be limited to requesting only mediaprograms offered by the media program provider.

In system 500, video data may be obtained from one or more sources forexample, from a video source 510, for use as input to a video contentserver 502. The input video data may comprise raw or edited frame-basedvideo data in any suitable digital format, for example, Moving PicturesExperts Group (MPEG)-1, MPEG-2, MPEG-4, VC-1, H.264/Advanced VideoCoding (AVC), High Efficiency Video Coding (HEVC), or other format. Inan alternative, a video may be provided in a non-digital format andconverted to digital format using a scanner and/or transcoder. The inputvideo data may comprise video clips or programs of various types, forexample, television episodes, motion pictures, and other contentproduced as primary content of interest to consumers. The video data mayalso include audio or only audio may be used.

The video streaming system 500 may include one or more computer serversor modules 502, 504, and/or 507 distributed over one or more computers.Each server 502, 504, 507 may include, or may be operatively coupled to,one or more data stores 509, for example databases, indexes, files, orother data structures. A video content server 502 may access a datastore (not shown) of various video segments. The video content server502 may serve the video segments as directed by a user interfacecontroller communicating with a client device. As used herein, a videosegment refers to a definite portion of frame-based video data, such asmay be used in a streaming video session to view a television episode,motion picture, recorded live performance, or other video content.

In some embodiments, a video advertising server 504 may access a datastore of relatively short videos (e.g., 10 second, 30 second, or 50second video advertisements) configured as advertising for a particularadvertiser or message. The advertising may be provided for an advertiserin exchange for payment of some kind, or may comprise a promotionalmessage for the system 500, a public service message, or some otherinformation. The video advertising server 504 may serve the videoadvertising segments as directed by a user interface controller (notshown).

The video streaming system 500 also may include image processor 110.

The video streaming system 500 may further include an integration andstreaming component 507 that integrates video content and videoadvertising into a streaming video segment. For example, streamingcomponent 507 may be a content server or streaming media server. Acontroller (not shown) may determine the selection or configuration ofadvertising in the streaming video based on any suitable algorithm orprocess. The video streaming system 500 may include other modules orunits not depicted in FIG. 5, for example administrative servers,commerce servers, network infrastructure, advertising selection engines,and so forth.

The video streaming system 500 may connect to a data communicationnetwork 512. A data communication network 512 may comprise a local areanetwork (LAN), a wide area network (WAN), for example, the Internet, atelephone network, a wireless cellular telecommunications network (WCS)514, or some combination of these or similar networks.

One or more client devices 520 may be in communication with the videostreaming system 500, via the data communication network 512 and/orother network 514. Such client devices may include, for example, one ormore laptop computers 520-1, desktop computers 520-2, “smart” mobilephones 520-3, tablet devices 520-4, network-enabled televisions 520-5,or combinations thereof, via a router 518 for a LAN, via a base station517 for a wireless telephony network 514, or via some other connection.In operation, such client devices 520 may send and receive data orinstructions to the system 500, in response to user input received fromuser input devices or other input. In response, the system 500 may servevideo segments and metadata from the data store 509 responsive toselection of media programs to the client devices 520. Client devices520 may output the video content from the streaming video segment in amedia player using a display screen, projector, or other video outputdevice, and receive user input for interacting with the video content.

Distribution of audio-video data may be implemented from streamingcomponent 507 to remote client devices over computer networks,telecommunications networks, and combinations of such networks, usingvarious methods, for example streaming. In streaming, a content serverstreams audio-video data continuously to a media player componentoperating at least partly on the client device, which may play theaudio-video data concurrently with receiving the streaming data from theserver. Although streaming is discussed, other methods of delivery maybe used. The media player component may initiate play of the video dataimmediately after receiving an initial portion of the data from thecontent provider. Traditional streaming techniques use a single providerdelivering a stream of data to a set of end users. High bandwidths andprocessing power may be required to deliver a single stream to a largeaudience, and the required bandwidth of the provider may increase as thenumber of end users increases.

Streaming media can be delivered on-demand or live. Streaming enablesimmediate playback at any point within the file. End-users may skipthrough the media file to start playback or change playback to any pointin the media file. Hence, the end-user does not need to wait for thefile to progressively download. Typically, streaming media is deliveredfrom a few dedicated servers having high bandwidth capabilities via aspecialized device that accepts requests for video files, and withinformation about the format, bandwidth and structure of those files,delivers just the amount of data necessary to play the video, at therate needed to play it. Streaming media servers may also account for thetransmission bandwidth and capabilities of the media player on thedestination client. Streaming component 507 may communicate with clientdevice 520 using control messages and data messages to adjust tochanging network conditions as the video is played. These controlmessages can include commands for enabling control functions such asfast forward, fast reverse, pausing, or seeking to a particular part ofthe file at the client.

Since streaming component 507 transmits video data only as needed and atthe rate that is needed, precise control over the number of streamsserved can be maintained. The viewer will not be able to view high datarate videos over a lower data rate transmission medium. However,streaming media servers (1) provide users random access to the videofile, (2) allow monitoring of who is viewing what video programs and howlong they are watched, (3) use transmission bandwidth more efficiently,since only the amount of data required to support the viewing experienceis transmitted, and (4) the video file is not stored in the viewer'scomputer, but discarded by the media player, thus allowing more controlover the content.

Streaming component 507 may use TCP-based protocols, such as HTTP andReal Time Messaging Protocol (RTMP). Streaming component 507 can alsodeliver live webcasts and can multicast, which allows more than oneclient to tune into a single stream, thus saving bandwidth. Streamingmedia players may not rely on buffering the whole video to providerandom access to any point in the media program. Instead, this isaccomplished through the use of control messages transmitted from themedia player to the streaming media server. Another protocol used forstreaming is hypertext transfer protocol (HTTP) live streaming (HLS) orDynamic Adaptive Streaming over HTTP (DASH). The HLS or DASH protocoldelivers video over HTTP via a playlist of small segments that are madeavailable in a variety of bitrates typically from one or more contentdelivery networks (CDNs). This allows a media player to switch bothbitrates and content sources on a segment-by-segment basis. Theswitching helps compensate for network bandwidth variances and alsoinfrastructure failures that may occur during playback of the video.

The delivery of video content by streaming may be accomplished under avariety of models. In one model, the user pays for the viewing of videoprograms, for example, using a fee for access to the library of mediaprograms or a portion of restricted media programs, or using apay-per-view service. In another model widely adopted by broadcasttelevision shortly after its inception, sponsors pay for thepresentation of the media program in exchange for the right to presentadvertisements during or adjacent to the presentation of the program. Insome models, advertisements are inserted at predetermined times in avideo program, which times may be referred to as “ad slots” or “adbreaks.” With streaming video, the media player may be configured sothat the client device cannot play the video without also playingpredetermined advertisements during the designated ad slots.

Referring to FIG. 6, a diagrammatic view of an apparatus 600 for viewingvideo content and advertisements is illustrated. In selectedembodiments, the apparatus 600 may include a processor (CPU) 602operatively coupled to a processor memory 604, which holds binary-codedfunctional modules for execution by the processor 602. Such functionalmodules may include an operating system 606 for handling systemfunctions such as input/output and memory access, a browser 608 todisplay web pages, and media player 610 for playing video. The modulesmay further include display process 108. The memory 604 may holdadditional modules not shown in FIG. 6, for example modules forperforming other operations described elsewhere herein.

A bus 614 or other communication component may support communication ofinformation within the apparatus 600. The processor 602 may be aspecialized or dedicated microprocessor configured to perform particulartasks in accordance with the features and aspects disclosed herein byexecuting machine-readable software code defining the particular tasks.Processor memory 604 (e.g., random access memory (RAM) or other dynamicstorage device) may be connected to the bus 614 or directly to theprocessor 602, and store information and instructions to be executed bya processor 602. The memory 604 may also store temporary variables orother intermediate information during execution of such instructions.

A computer-readable medium (CRM) in a storage device 624 may beconnected to the bus 614 and store static information and instructionsfor the processor 602; for example, the storage device (CRM) 624 maystore the modules 606, 608, and 610 when the apparatus 600 is poweredoff, from which the modules may be loaded into the processor memory 604when the apparatus 600 is powered up. The storage device 624 may includea non-transitory computer-readable storage medium holding information,instructions, or some combination thereof, for example instructions thatwhen executed by the processor 602, cause the apparatus 600 to beconfigured to perform one or more operations of a method as describedherein.

A communication interface 616 may also be connected to the bus 614. Thecommunication interface 616 may provide or support two-way datacommunication between the apparatus 600 and one or more externaldevices, e.g., the streaming system 500, optionally via a router/modem626 and a wired or wireless connection. In the alternative, or inaddition, the apparatus 600 may include a transceiver 618 connected toan antenna 629, through which the apparatus 600 may communicatewirelessly with a base station for a wireless communication system orwith the router/modem 626. In the alternative, the apparatus 600 maycommunicate with a video streaming system 500 via a local area network,virtual private network, or other network. In another alternative, theapparatus 600 may be incorporated as a module or component of the system500 and communicate with other components via the bus 614 or by someother modality.

The apparatus 600 may be connected (e.g., via the bus 614 and graphicsprocessing unit 620) to a display unit 628. A display 628 may includeany suitable configuration for displaying information to an operator ofthe apparatus 600. For example, a display 628 may include or utilize aliquid crystal display (LCD), touchscreen LCD (e.g., capacitivedisplay), light emitting diode (LED) display, projector, or otherdisplay device to present information to a user of the apparatus 600 ina visual display.

One or more input devices 630 (e.g., an alphanumeric keyboard,microphone, keypad, remote controller, game controller, camera or cameraarray) may be connected to the bus 614 via a user input port 622 tocommunicate information and commands to the apparatus 600. In selectedembodiments, an input device 630 may provide or support control over thepositioning of a cursor. Such a cursor control device, also called apointing device, may be configured as a mouse, a trackball, a track pad,touch screen, cursor direction keys or other device for receiving ortracking physical movement and translating the movement into electricalsignals indicating cursor movement. The cursor control device may beincorporated into the display unit 628, for example using a touchsensitive screen. A cursor control device may communicate directioninformation and command selections to the processor 602 and controlcursor movement on the display 628. A cursor control device may have twoor more degrees of freedom, for example allowing the device to specifycursor positions in a plane or three-dimensional space.

Particular embodiments may be implemented in a non-transitorycomputer-readable storage medium for use by or in connection with theinstruction execution system, apparatus, system, or machine. Thecomputer-readable storage medium contains instructions for controlling acomputer system to perform a method described by particular embodiments.The computer system may include one or more computing devices. Theinstructions, when executed by one or more computer processors, may beconfigured to perform that which is described in particular embodiments.

As used in the description herein and throughout the claims that follow,“a”, “an”, and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

The above description illustrates various embodiments along withexamples of how aspects of particular embodiments may be implemented.The above examples and embodiments should not be deemed to be the onlyembodiments, and are presented to illustrate the flexibility andadvantages of particular embodiments as defined by the following claims.Based on the above disclosure and the following claims, otherarrangements, embodiments, implementations and equivalents may beemployed without departing from the scope hereof as defined by theclaims.

What is claimed is:
 1. A method comprising: receiving, by a computingdevice, an image of video content for a video offered by a videodelivery service; analyzing, by the computing device, at least a portionof pixels in the image to determine color attributes of the at least theportion of the pixels, the color attributes including a lightnessattribute and a hue attribute; selecting, by the computing device, alightness classification from a plurality of lightness classificationsbased on the lightness attribute and selecting a hue groupclassification from a plurality of hue group classifications based onthe hue attribute; storing, by the computing device, the lightnessclassification and the hue group classification with the image; andstoring, by the computing device, a plurality of swatch configurationsfor the plurality of lightness classifications, wherein the plurality ofswatch configurations define different saturation values and differentlightness values to use for a hue group classification, and wherein theimage, the lightness classification, the plurality of swatchconfigurations, and the hue group classification are sent to a clientdevice to allow the client device to compute a plurality of colors usinga selected swatch configuration from the plurality of swatchconfigurations that is associated with the lightness classification togenerate a plurality of combinations for lightness values, saturationvalues, and hue values by varying at least one of the saturation valuedefined by the selected swatch configuration, the lightness valuedefined by the selected swatch configuration, and a hue value defined bythe hue group classification without analyzing the image to determinethe lightness classification and the hue group classification, theplurality of colors being used to accent one or more accent portionsincluded in an interface that is generated on the client device todisplay the image.
 2. The method of claim 1, further comprising: sendingthe saturation value to the client for use in generating the pluralityof colors, wherein the saturation value is pre-defined.
 3. The method ofclaim 1, wherein the swatch configuration defines a plurality oflightness values and a plurality of saturation values for the pluralityof lightness classifications, wherein a lightness value and a saturationvalue are selected by the client device based on the lightnessclassification received for the image.
 4. The method of claim 1, whereinthe plurality of swatch configurations are used to generate a pluralityof combinations for lightness values, saturation values, and hue valuesfor the lightness classification and the hue group classification fordifferent images.
 5. The method of claim 1, further comprising:generating a set of lightness counts for the image based on thelightness attribute and a set of hue group counts based on the hueattribute for the at least a portion of the pixels, wherein thelightness classification is determined from the set of lightness countsand the hue group classification is determined from the set of hue groupcounts.
 6. The method of claim 5, wherein generating the set oflightness counts comprises: for each pixel in the at least the portionof the pixels, performing: determining a lightness value from thelightness attribute for the respective pixel; comparing the lightnessvalue to a set of thresholds; and incrementing one of the set oflightness counts based on the comparing.
 7. The method of claim 6,wherein comparing the lightness value comprises: when the lightnessvalue is less than or equal to a first threshold, incrementing a firstlightness count; when the lightness value is greater than the firstthreshold and less than a third threshold, incrementing a secondlightness count; and when the lightness value is greater than the thirdthreshold, incrementing a third lightness count.
 8. The method of claim7, wherein: the first lightness count is a dark classification, thesecond lightness count is a medium classification; and the thirdlightness count is a light classification.
 9. The method of claim 5,wherein generating the set of hue group counts comprises: for one ormore pixels in the at least the portion of the pixels, performing:determining a hue group in which the hue attribute belongs for therespective pixel; and incrementing the hue group count for the huegroup.
 10. The method of claim 6, wherein generating the set of huegroup counts comprises: for one or more pixels in the at least theportion of the pixels, performing: determining when the respective pixelshould not be considered for incrementing any of the set of hue groupcounts; and not incrementing any of the set of hue group counts for therespective pixel.
 11. The method of claim 10, wherein one or more pixelsin the at least the portion of the pixels are not considered forincrementing any of the set of hue group counts when: a chroma attributeof the respective pixel falls outside of a chroma range; a saturationattribute of the respective pixel falls outside of a saturation range;the lightness attribute of the respective pixel falls outside of alightness range; and a maximum distance between red, green, and bluecomponents of the respective pixel is within a distance threshold. 12.The method of claim 6, wherein a second portion of pixels of the imageare not analyzed and not used to generate the set of lightness countsand set of hue group counts.
 13. A non-transitory computer-readablestorage medium containing instructions, that when executed, control acomputer system to be configured for: receiving an image of videocontent for a video offered by a video delivery service; analyzing atleast a portion of pixels in the image to determine color attributes ofthe at least the portion of the pixels, the color attributes including alightness attribute and a hue attribute; selecting a lightnessclassification from a plurality of lightness classifications based onthe lightness attribute and selecting a hue group classification from aplurality of hue group classifications based on the hue attribute;storing the lightness classification and the hue group classificationwith the image; and storing a plurality of swatch configurations for theplurality of lightness classifications, wherein the plurality of swatchconfigurations define different saturation values and differentlightness values to use for a hue group classification, and wherein theimage, the lightness classification, the plurality of swatchconfigurations, and the hue group classification are sent to a clientdevice to allow the client device to compute a plurality of colors usinga selected swatch configuration from the plurality of swatchconfigurations that is associated with the lightness classification togenerate a plurality of combinations for lightness values, saturationvalues, and hue values by varying at least one of the saturation valuedefined by the selected swatch configuration, the lightness valuedefined by the selected swatch configuration, and a hue value defined bythe hue group classification without analyzing the image to determinethe lightness classification and the hue group classification, theplurality of colors being used to accent one or more accent portionsincluded in an interface that is generated on the client device todisplay the image.
 14. The non-transitory computer-readable storagemedium of claim 13, further configured for: sending a swatchconfiguration that defines a plurality of lightness values and aplurality of saturation values for the plurality of lightnessclassifications, wherein a lightness value and a saturation value areselected by the client device based on the lightness classificationreceived for the image.
 15. The non-transitory computer-readable storagemedium of claim 13, further comprising: generating a set of lightnesscounts for the image based on the lightness attribute and a set of huegroup counts based on the hue attribute for the at least a portion ofthe pixels, wherein the lightness classification is determined from theset of lightness counts and the hue group classification is determinedfrom the set of hue group counts.
 16. The non-transitorycomputer-readable storage medium of claim 15, wherein generating the setof lightness counts comprises: for each pixel in the at least theportion of the pixels, performing: determining a lightness value fromthe lightness attribute for the respective pixel; comparing thelightness value to a set of thresholds; and incrementing one of the setof lightness counts based on the comparing.
 17. The non-transitorycomputer-readable storage medium of claim 15, wherein generating the setof hue group counts comprises: for each pixel in the at least theportion of the pixels, performing: determining a hue group in which thehue attribute belongs for the respective pixel; and incrementing the huegroup count for the hue group.
 18. A method comprising: determining, bya computing device, a plurality of swatch configurations that definedifferent saturation values and lightness values to use for a pluralityof lightness classifications; receiving, by the computing device, animage of video content for a video offered by a video delivery service;receiving, by the computing device, a lightness classification from theplurality of lightness classifications and a hue group classificationfrom a plurality of hue group classifications; selecting, by thecomputing device, a swatch configuration from the plurality of swatchconfigurations, wherein the swatch configuration that defines aplurality of combinations for lightness values, saturation values andhue values for the lightness classification and the hue groupclassification; computing, by the computing device, a plurality ofcolors using the plurality of combinations for lightness values, theplurality of saturation values, and hue values by varying at least oneof the saturation value defined by the selected swatch configuration,the lightness value defined by the selected swatch configuration, and ahue value defined by hue group classification without analyzing theimage to determine the lightness classification and the hue groupclassification; and generating, by the computing device, an interface inwhich the image is displayed, the interface including one or more accentportions that are displayed with different images, wherein the pluralityof colors are used to accent the one or more accent portions of thegenerated interface and the one or more accent portions are accentedwith different colors when different images are displayed in theinterface.
 19. The method of claim 18, further comprising: receiving anavigation to an entity displayed on the interface; and displaying theimage and the applied plurality of colors to the interface in responseto receiving the navigation.
 20. The method of claim 18, furthercomprising: using the lightness classification to determine one or morelightness values and one or more saturation values for the plurality ofcombinations; and using the hue group classification to determine one ormore hue values for the plurality of combinations.
 21. The method ofclaim 18, wherein the swatch configuration is received from the videodelivery service.
 22. The method of claim 18, wherein the plurality ofswatch configurations are used to generate a plurality of combinationsfor lightness values, saturation values, and hue values for thelightness classification and the hue group classification for thedifferent images.